Intro

I am a Research Scientist at Google Deepmind working in the Fundamental Research Org. Before this, I was a PhD student in the Computer Science Department at Carnegie Mellon University co-advised by Ameet Talwalkar and Graham Neubig. Before starting my PhD, I was a Research Engineer at Facebook AI Research (FAIR) in Seattle. My current research interests broadly cover all things related to learning from multiple tasks: transfer learning, meta-learning, multi-tasking and auxiliary learning. I primarily explore these fields in the context of Natural Language Processing but the tools I develop are domain agnostic. I have previously worked in Computer Vision (CV) - specifically CV algorithms for High Energy Physics (HEP) and Video Understanding. I did my undergrad (Physics) and Masters (Computer Science) at Stanford University. Outside of A.I, I am very passionate about Reading, Music, Art, Neuroscience, Philosophy and Food !!.

2024 Book List

  • How not to be wrong - Jordan Ellenberg
  • The ones who walk away from omelas - Ursula K. Le Guin
  • Algorithms to live by - Brian Christian, Thomas L. Griffiths
  • Winter's Heart: Wheel of Time Series - Robert Jordan
  • Crossroads of Twilight: Wheel of Time Series - Robert Jordan
  • The Dark Forest (Three Body Problem - Book II) - Liu Cixin
  • Death's End (Three Body Problem - Book III) - Liu Cixin
  • Parable of the Sower - Octavia Butler

In Progress

  • The Fall - Albert Camus
  • Special Relativity andd Classical Field Theory - Leonard Susskind and Art Friedman
  • The Sandman (Book 3) - Neil Gaiman
  • I Will Teach You To Be Rich - Ramit Sethi
  • Collected Fictions of Jorge Luis Borges

2023 Book List

  • The Alchemist - Paulo Coelho
  • Ikigai - Hector Garcia and Francesc Miralles
  • Thinking Fast and Slow - Daniel Kahneman
  • Stories of Your Life and Others - Ted Chiang

2022 Book List

  • The Path of Daggers - WOT Series - Robert Jordan
  • The Hard Thing about Hard Things - Ben Horowitz
  • The Everything Store - Brad Stone
  • The Alignment Problem - Brian Christian
  • Poorly Understood: What America gets wrong about poverty - Heather E. Bullock et Al
  • Race After Technology - Ruha Benjamin
  • The Graveyard Book - Neil Gaiman
  • Dreams of a Final Theory - Steven Weinberg
  • Chaos - James Gleick
  • Existential Physics - Sabine Hossenfelder
  • The Selfish Gene - Richard Dawkins

2021 Book List

  • The Sandman Act I & II - Neil Gaiman
  • A Crown of Swords - WOT Series - Robert Jordan
  • Intimations - Zadie Smith
  • Project Hail Mary - Andy Weir
  • Hold Everything Dear - John Berger
  • Guns, Germs and Steel - Jared Diamond
  • Hyperbole and Half - Allie Brosh
  • Ministry For The Future - Kim Stanley Robinson

2020 Book List

  • The Stranger - Albert Camus
  • Awards for good boys - Shelby Lorman
  • The Game - Neil Strauss
  • Predictably Irrational - Daniel Ariely
  • The Genealogy of Morals - Friedrich Nietzsche
  • The Three-Body Problem - Cixin Liu
  • Atomic Habits - James Clear
  • Modern Ethics in 77 Arguments : A Stone Reader
  • Dune - Frank Herbert

2019 Book List

  • Man's Search For Meaning - Viktor Frankl
  • Reductionism in Art and Brain Science - Eric R. Kandel
  • The Myth of Sisyphus - Albert Camus
  • Logic Comix - Apostolos Doxiadis, Christos Papadimitriou
  • The New Jim Crow - Michelle Alexander
  • The Fifth Season - N.K. Jemisin
  • Deep Work - Cal Newport
  • The Riddle Master Series [3 Books] - Patricia McKillip
  • His Dark Materials [3 Books] - Phillip Pullman
  • The Upward Spiral - Alex Korb
  • Portraits of Resilience - Daniel Jackson

Full CV

Education

Aug 2019 - July 2024 PhD in Computer Science
Carnegie Mellon University
Sept 2016 - June 2018 Masters in Computer Science ** Tau Beta Pi
Stanford University
Sept 2013 - June 2018 Bachelors in Physics, Minor in Computer Science ** with Distinction
Stanford University

Experience

August 2024 - Research Scientist , Google DeepMind
May 2023 - August 2023 AI/ML Research Intern, Apple
March 2023 - May 2023 Student Researcher, Google Research
May 2022 - August 2022 Research Scientist Intern, DeepMind
July 2020 - August 2020 Research Scientist Intern, Google Brain - Google
July 2018 - July 2019 Research Engineer, Facebook A.I Research - Facebook
June 2017 - Sept 2017 Software Engineering Intern, Applied Machine Learning - Facebook
June 2016 - Sept 2016 Software Engineering Intern, Terra Bella - Google
June 2015 - Sept 2015 Software Engineering Intern, Google Analytics - Google

2024

Everybody Prune Now: Structured Pruning of LLMs with only Forward Passes
Lucio Dery, Steven Kolawole, Jean-François Kagy, Virginia Smith, Graham Neubig, Ameet Talwalkar
Preprint (under submission)
On Resource Efficient Transfer Learning via End Task Aware Training
Lucio Dery
PhD Thesis
DeMuX: Data-efficient Multilingual Learning
Simran Khanuja, Srinivas Gowriraj, Lucio Dery, Graham Neubig
NAACL 2024
Multitask Learning can improve robustness to worst group outcomes
Atharva Kulkarni*, Lucio Dery*, Amrith Setlur, Aditi Raghunathan, Ameet Talwalkar, Graham Neubig
TMLR 2024
SSL Theory and Practice Workshop NeurIPS 2023

2023

Tranfer Learning for Structured Pruning under Limited Data
Lucio Dery, Awni Hannun, David Grangier
ENLSP Workshop NeurIPS 2023
Cross-Modal Fine-Tuning: Align then Refine
Junhong Shen, Liam Li, Lucio Dery, Corey Staten, Mikhail Khodak, Graham Neubig, Ameet Talwalkar.
ICML 2023 (Oral)
AANG: Automating Auxiliary Learning
Lucio Dery, Paul Michel, Mikhail Khodak, Graham Neubig, Ameet Talwalkar.
ICLR 2023 (Spotlight)

2022

Multi-step Planning for Automated Hyperparameter Optimization with OptFormer
Lucio Dery, Abram L. Friesen, Nando De Freitas, Marc'Aurelio Ranzato, Yutian Chen
Workshop on Foundation Models For Decision Making - NeurIPS 2022
Should we be Pre-Training ? An Argument for End-Task Aware Training as an Alternative
Lucio Dery, Paul Michel, Ameet Talwalkar, Graham Neubig
ICLR, 2022

2021

Auxiliary Task Update Decomposition : The Good, The Bad and The Neutral
Lucio Dery, David Grangier, Yann Dauphin
ICLR, 2021

2018

Audio to Body Dynamics
Eli Shlizerman, Lucio Dery, Hayden Schoen, Ira Kemelmacher. “Audio to Body Dynamics.”
CVPR, 2018
Finding ‘It’: Weakly-Supervised Reference-Aware Visual Grounding in Instructional Video
D.A-Huang, Shyamal Buch, Lucio Dery, Animesh Garg, Li Fei-Fei, Juan Carlos Niebles
CVPR, 2018 (Oral)

2017

Weakly supervised classification in high energy physics
Lucio Dery , Benjamin Nachman, Francesco Rubbo, Ariel Schwartzman
Journal of High Energy Physics 2017.5 (2017): 1-11

Teaching

Fall 2023 Teaching Assistant, Machine learning with large datasets (10-605) Carnegie Mellon University
Fall 2022 Teaching Assistant, Advanced Natural Language Processing (11-711) Carnegie Mellon University
Winter 2018 Head Teaching Assistant, Deep Learning (CS230) Stanford University
Spring 2018 Course Assistant, Deep Learning (CS230) Stanford University
Fall 2017 Course Assistant, Machine Learning (CS229) Stanford University
Winter 2014 - Spring 2017 Section Leader, Programming Abstactions (CS106A) Stanford University
Winter 2014 - Spring 2017 Section Leader, Programming Methodology (CS106B) Stanford University

Email :

ldery [at] andrew [dot] cmu [dot] edu

Working on getting better at drawing and painting ! I hope I find the time. Here's some random artwork I've made in the past

Art